Prior art network management systems collected network system event information in an ASCII text file, typically in a serial data stream format.
Network system managers often require statistics and reports based on collected system events to alert them to system trends in order to proactively maintain normal system usage. Generally, this information is derived from the serial ASCII text event file by external statistical process control (SPC) and trend tracking applications, collectively referred herein as report generating applications, that gather, manipulate and format information from the event file to generate user-readable reports. However, in the past, each report generating application had to know the system-specific format of the contents of the event file in order to derive the required information to generate reports. Accordingly, a report generating application had to know the system specific format of event data stored in the event file, either by separately compiling the application with the system specific event file format or by feeding it event format information at setup time or runtime. Thus, in order to utilize such an application, the application developer had to provide some means of supporting the event file format. Accordingly, a need exists in the industry for a formal standardized data schema, a method for populating it, and a way to easily extract information from which various reports are generated.
In addition, due to breadth of different categories of system events, not all system events included the same amount of information. In some prior art implementations, this problem has been dealt with by allocating a standard size system event packet for any type system event, and allowing the field definitions to vary within the packet for each event type to accommodate the differing reporting information for different type system events. While this technique provides some leeway in the type of information reported in differing types of system events, it requires the standard size of a system event packet to be large enough to store a system event of the type having system event information of greatest length, and hence results in significant memory inefficiency due to allocated but unused bytes in event packets for those system events that do not require the full size of the standard size system event packet. Accordingly, a need exists for a method and data schema for storing system events dynamically according to the amount of required storage space.
One method for overcoming this problem is by defining a standard size system event packet that includes only those fields that are common to all events, and linking it to a variable binding packet that may vary in size according to the system event type. This technique allows the number and definitions of event fields to vary between different system event types, and ensures maximum memory efficiency. However, data required to generate useful reports about the system events is nonetheless difficult to extract. Report generating applications organize event information according to one or a few categories at a time, and then summarize information around those categories in a manner useful to the system administrator. In generating a report, the application typically utilizes in only a small subset of all of the detailed system event data for each event. In the prior art event storage techniques described above, however, the extraction of the relevant subset of fields from the complete detailed set of event fields requires a separate query on the database for each required event field for each event. When processing large amounts of event data and or when the application requires more than a small few event fields for each event, the high volume of queries on the database becomes a performance drain on the system. Accordingly, a need exists for a method for storing event data in a manner that allows relevant information for report generating applications to be easily extractable.